Introspective Learning by Distilling Knowledge from Online Self-explanation

In recent years, many methods have been proposed to explain individual classification predictions of deep neural networks. However, how leverage the created explanations improve learning process has less explored. The extracted from a model can be used guide itself. Another type information training is knowledge provided by powerful teacher model. goal this work self-explanation borrowing ideas distillation. We start investigating effective components transferred network student network. Our investigation reveals that both responses in non-ground-truth classes and class-similarity teacher’s outputs contribute success Motivated conclusion, we propose an implementation introspective distilling online self-explanations. models trained with procedure outperform ones standard procedure, as well different regularization methods. When compared learned peer networks or networks, our also show competitive performance requires neither peers nor teachers.